Gaussian Mixure Models and Expectation Maximization

The goal of the assignment is to use the Expectation Maximization (EM) algorithm to estimate the parameters of a two-component Guassian Mixture in two dimensions. This involves estimating the mean vector μk and covariance matrix Σk for both distributions as well as the mixing coefficients (or prior probabilities) πk for each component k. EM works by first choosing an arbitrary parameter set. In my implementation, I sampled the initial means from a zero-mean unit variance normal distribution, and the covariance matrices were initially set to the identity. I set π1 to a random value between zero and one and set π2 = 1− π1. EM proceeds by estimating the ”responsibilities” of each component for each data point in the E-step. The means, covariance, and mixing coefficients for both components are then re-estimated using the data weighted by the responsibility values in the M-step. This process alternates until a halting criterion is reached. I used the change in the log likelihood as my halting criterion and ran the algorithm until the change in log likelihood was less than = 0.005.